Stopping criteria for, and strong convergence of, stochastic gradient descent on Bottou-Curtis-Nocedal functions

Stopping criteria for Stochastic Gradient Descent (SGD) methods play important roles from enabling adaptive step size schemes to providing rigor downstream analyses such as asymptotic inference. Unfortunately, current stopping SGD are often heuristics that rely on normality results or convergence stationary distributions, which may fail exist nonconvex functions and, thereby, limit the applicability of criteria. To address this issue, in work, we rigorously develop two can be applied a broad class functions, term Bottou-Curtis-Nocedal functions. Moreover, prerequisite developing these criteria, prove gradient function evaluated at SGD’s iterates converges strongly zero addresses an open question literature. As result our developed used new bolster other